Reflective memory bridge for external computing nodes

ABSTRACT

In at least some examples, a computing node includes a processor and a local memory coupled to the processor. The computing node also includes a reflective memory bridge coupled to the processor. The reflective memory bridge maps to an incoming region of the local memory assigned to at least one external computing node and maps to an outgoing region of the local memory assigned to at least one external computing node.

BACKGROUND

Database systems are evolving from disk-based single server systems withhigh input/output (I/O) rates and small memories, to clusters ofindependent nodes which hold the database entirely in memory. Thesesystems often rely on horizontal partitioning (or sharing) for scalingand k-safety (synchronous replication) to provide durability.

Data contention and metadata contention are ongoing challenges foronline transaction processing (OLTP) systems. The probability ofcontention increases as the number of parallel transactions in thesystem increases. Transaction response time is linked to the number ofactive parallel transactions by Little's law. This law states that thenumber of transactions in a system equals the throughput multiplied bythe response time.

BRIEF DESCRIPTION OF THE DRAWINGS

For a detailed description of illustrative examples of the disclosure,reference will now be made to the accompanying drawings in which;

FIG. 1 shows an example of a computing node system in accordance withthe disclosure;

FIGS. 2A-2D show other examples of computing node systems in accordancewith the disclosure;

FIG. 3 shows an example of an integrated circuit in accordance with thedisclosure;

FIG. 4 shows an example of a monitoring system fora computing node inaccordance with the disclosure;

FIG. 5 shows an example of another monitoring system for a computingnode in accordance with the disclosure;

FIG. 6 shows an example of a computer system in accordance with thedisclosure; and

FIG. 7 shows an example of a method in accordance with the disclosure.

NOTATION AND NOMENCLATURE

Certain terms are used throughout the following description and claimsto refer to particular system components. As one skilled in the art willappreciate, computer companies may refer to a component by differentnames. This document does not intend to distinguish between componentsthat differ in name but not function. In the following discussion and inthe claims, the terms “including” and “comprising” are used in anopen-ended fashion, and thus should be interpreted to mean “including,but not limited to . . . .” Also, the term “couple” or “couples” isintended to mean either an indirect, direct, optical or wirelesselectrical connection. Thus, if a first device couples to a seconddevice, that connection may be through a direct electrical connection,through an indirect electrical connection via other devices andconnections, through an optical electrical connection, or through awireless electrical connection.

DETAILED DESCRIPTION

The following discussion is directed to methods and systems foremploying reflective memory in a multi-node computing and databaseenvironment. In one example, a computing node may include local memoryand a reflective memory bridge (controller) that maps to an incomingregion of the local memory assigned to at least one external computingnode and that maps to an outgoing region of the local memory assigned toat least one external computing node. The reflective memory bridge maybe integrated with another component such as a processor, a nodecontroller, a coherency bus, or a peripheral component interconnect(PCI) card. As used herein, an “external computing node” refers to acomputer running a different instance of an operating system and notnecessarily running the same operating system or using the same CPUarchitecture. As used herein, “reflective memory” refers to a mechanismfor snarling data written to one or more regions of memory of a node. Inthe disclosed examples, when a store falls within a particular range ofaddresses, a copy of the data is forwarded to at least one externalcomputing node. The external node's reflective memory controller storesthe write data in that node's memory at a relative memory location thatcorresponds to the particular range from which the data was sent.

In the disclosed examples, reflective memory is used to maintain acoherent copy of a memory region in one node or multiple nodes in orderto maintain the database concept of k-safety. Examples of applicationswhich use k-safety are Vertica and Hadoop. The disclosed examplesmaintain the concept of k-safety with minimum latency and withoutrequiring input/output path instructions to be executed on the nodesinvolved. The interconnect between nodes does not have to be areflective memory bus, but rather can be any interconnect including anetwork. The network should maintain the order of forwarded stores fromeach node-region pair to each particular node. No other ordering isnecessary (e.g., no ordering between regions is needed).

Without limitation to other examples, the reflective memory bridge mayspecify for an outgoing region: a name, addresses associated with theoutgoing region, and a set of external computing nodes that are toreceive updates to the outgoing region. Further, the reflective memorybridge may specify for an incoming region: a name, addresses associatedwith the incoming region, and a set of external computing nodes that areto reflect to the incoming region. In some examples, a reflective memorybridge may specify a plurality of different outgoing regions and/orincoming regions. Further, a reflective memory bridge may update theplurality of different outgoing regions and/or incoming regions uponrequest from a user or administrator. Further, a reflective memorybridge may be part of with a coherency fabric and/or an interconnectmonitoring interface of a computing node.

In some examples, the addresses corresponding to the incoming regionsand outgoing regions are physical memory addresses. In other examples,snarfing may occur pre-V2P (virtual-to-physical), and a field in thevirtual address would contain the remote node ID or a broadcast key.Likewise, an incoming transaction would have a virtual address whichwould get protection-checked and translated by the receiving core justlike a store that originated from its own pipeline.

In some examples, a computing node includes a database with access tothe reflective memory bridge. Without limitation to other examples, thedatabase may utilize the reflective memory bridge to perform operationssuch as communicating commands or results to at least one externalcomputing node, distributing logs, updates, and new data to at least oneexternal computing node, communicating information for two-phase commitsto at least one other computing node, maintaining k-safety by reflectingat least part of a projection being written to a mirror node, and/orwriting to a logging appliance.

The disclosed examples for employing reflective memory in a multi-nodecomputing and database environment differ from message-based systemsthat transmit messages with a contiguous buffer over a network, orsystems that rely on a shared memory paradigm. While the shared memoryparadigm is easy to program and is efficient, shared memory systems aresubject to single points of failure that can bring down all systemsaccessing the shared memory region. Meanwhile, message-based systemsrequire buffers to be constructed after which the message is sent to theother system. Because no part of the message is sent until the bufferfilling is completed, significant instruction path length increases mayoccur during I/O operations and may interrupt processing. Use ofreflective memory operations to reduce transaction response time acrossmultiple computing nodes as disclosed reduces lock and latch relatedcontention and in the end helps to further reduce response time.

The disclosed examples for employing reflective memory in a multi-nodecomputing and database environment should not be interpreted, orotherwise used, as limiting the scope of the disclosure, including theclaims. in addition, one skilled in the art will understand that thefollowing description has broad application, and the discussion of anyparticular example is not intended to intimate that the scope of thedisclosure, including the claims, is limited to that example.

FIG. 1 shows an example of a computing node system 100 in accordancewith the disclosure. As shown, the computing node system 100 comprises acomputing node 101 in communication with external computing nodes 112.More specifically, the computing node 101 comprises a processor 102coupled to a local memory 104 with an incoming region 106 and anoutgoing region 108 associated with a reflective memory bridge 110. Thereflective memory bridge 110 maps to the incoming region 106, which isassigned to at least one external computing node 112. The reflectivememory bridge 110 also maps to the outgoing region 106, which isassigned to at least one external computing node 112. The externalcomputing nodes 112 corresponding to the incoming region 106 may be thesame or may be different than the external computing nodes 112corresponding to the outgoing region 108. Although not required, thecomputing node 101 may optionally include input/output (I/O) interfacecomponents (not shown for convenience).

For the incoming region 106, the reflective memory bridge 110 specifiesa name or identifier, physical addresses associated with the incomingregion 106, and a set of external computing nodes 112 that are toreflect to the incoming region 106. For the outgoing region 108, thereflective memory bridge 110 specifies a name or identifier, physicaladdresses associated with the outgoing region 108, and a set of externalcomputing nodes that are to receive updates to the outgoing region 108.In at least some examples, stores from the reflective memory bridge 110are not reflected. More specifically, stores generated by a destinationnode will not be reflected. Thus, two nodes are able to reflect the samememory locations to each other (similar to shared memory) withoutrisking an infinite loop. In other examples, regions are set up so thatnode A reflects to node B, and node B reflects the region to node C. Inanother example, node A reflects to nodes B and C.

In some examples, the reflective memory bridge 110 corresponds to anapplication-specific integrated circuit (ASIC) configured to perform thereflective memory operations described herein. Alternatively, thereflective memory bridge 110 may correspond to a processor executingreflective memory bridge instructions stored in a memory available tothe processor. Alternatively, the reflective memory bridge 110 maycorrespond to programmable logic configured to perform the reflectivememory operations described herein. The decisions regarding how toimplement the reflective memory bridge may be based, for example, onperformance criteria, cost criteria, and/or compatibility criteria. Forall implementations, the reflective memory bridge 110 needs to becompatible with an existing or planned coherency fabric of a computingnode.

In operation, the reflective memory bridge 110 observes all stores tothe outgoing region 108 and transmits those stores to others of theexternal computing nodes 112. The external computing nodes 112 may eachcomprise their own local reflective memory bridge to repeat those storesto local memory. In this manner, a message or change in state istransmitted as it is composed rather than waiting to transmit the firstbyte until the last byte is copied into a buffer as is done forbuffer-based systems. Also, there is no requirement that the words in amessage be written in order. To be clear, while there is no requirementthat the words in a region be written in any particular order, the wordswritten from a node to the region will be reflected in order to allsharing nodes. If two nodes are writing to the same region all sharingnodes may not see those writes in the same order. It is theresponsibility of reflective memory bridge software to deal with thispossibility. Single writer multiple readers are able to see stores inthe same order.

Thus, while traditional message passing waits to begin transmissionuntil the last store to a buffer is completed, reflective memoryoperations implemented by the reflective memory bridge 110 allows eachstore to be sent immediately (i.e., message latency is reduced).

The disclosed approach differs from traditional sender-managedcommunications (e.g. remote direct memory access or ROMA) in that thereis no need to build a buffer, nor execute a protocol stack, nor field aninterrupt (these operations may still be optionally done). This permitsflexible zero copy solutions.

In the disclosed computing node system 100, no memory is shared andthere are no multi-node single points of failure. In other words, eachcomputing node (e.g., computing nodes 101 and 112) has its own localmemory into which updates are copied, and none of the computing nodes101 and 112 reads another's local memory. Readers and writers may fail,but this will not cause a hardware crash in other nodes. Also, since aregion of memory is watched for update, it is possible to keep datastructures update in situ, and not require a buffer to be created. Thisallows scatter-scatter communication. Although only one incoming region106 and one outgoing region 108 are shown in FIG. 1, additional incomingregions and/or outgoing regions may be defined, where each region canindependently be reflected to a different set of servers correspondingto the external computing nodes 112.

To set up reflective memory operation, each computing node (e.g.,computing nodes 101 and 112) of the computing node system 100 maycommunicate with its respective reflective memory bridge through amemory mapped region. This region is used to set up one or more outgoingmemory regions to be reflected. In some examples, for each outgoingregion the following is specified: a name for the region (if empty thisregion is not active), physical addresses that contribute to the region,and a set of nodes that will receive updates to this region. Similarly,for each incoming region, the following may specified: a name for theregion (matched to incoming regions or empty if this region is notactive), physical addresses that comprise the region, and a set of nodeswhich may reflect to this region (the nodes may be checked to reduceerrors). The specified incoming regions will receive data reflected froma sender node, where the mapping of addresses is relative to the firstbyte of each region.

Additional extensions to the computing node system 100 may include atleast one memory mapped register that tracks a count of how many writesof a region are still in hardware buffers and have not yet beentransferred. The memory mapped register(s) is incremented whenever awrite is observed and decremented when a write has been sent to theexternal computing nodes 112. In this manner, a sender can be sure thata message has been sent. Along the same lines, an interrupt could beraised if the hardware buffers have reached a threshold fill level,indicating a possible interconnect problem.

For some examples of the computing node system 100, there may be a needfor safe and efficient synchronization. Toward that end, memory mappedinterfaces could be used to allow remote read/exchange constructs and/orread/increment constructs with timeout/error indications in the eventthat the memory is unavailable. In some examples, one of the Lamportalgorithms may be used for high-priority sections. Without limitation toother examples, the reflective memory bridge 110 may enable thecomputing node 101 to communicate commands and results to at least oneof the external computing nodes 112, to distribute logs, updates and newdata across the external computing nodes 112, to communicate informationfor two-phase commits to at least one of the external nodes 112, tomaintain k-safety by reflecting the part of a projection that is beingwritten to at least at least one of the external computing nodes 112operating as a mirror node for the computing node 101, and to write to alogging appliance corresponding to at least one of the externalcomputing nodes 112.

Use of the reflective memory bridge as described herein reducescommunications related latencies significantly and requires no driveroverhead. Thus, there is little to no impact on instruction path lengthand building of a contiguous buffer is avoided. Further, zero copy ispossible, Further, multi-node single points of failure are avoided.Further, in situ synchronization of data structures is possible.Further, by reducing communications latency the disclosed reflectivememory operations enable the highest scaling of horizontally partitioneddatabases.

FIGS. 2A-2D show other examples of computing node systems 200A-200D inaccordance with the disclosure. As shown, in the computing node system200A of FIG. 2A, the computing node 201A comprises a processor 202 withthe reflective memory bridge 110 integrated therein. In the computingnode 201A, a coherency fabric 212 enables reflective memory operationsinvolving the processor 202, the local memory 104, and the externalcomputing nodes 112. Alternatively, the reflective memory bridge 110 maybe implemented off of the coherency fabric 212 or directly from thebridge, or from the bridge to an I/O device to an interconnect that isnot the coherency fabric 212.

Further, a database 204 of the computing node 201A is able to requestthat the reflected memory controller 110 perform reflective memoryoperations to communicate commands and results to at least one of theexternal computing nodes 112, to distribute logs, updates and new dataacross the external computing nodes 112, to communicate information fortwo-phase commits to at least one of the external nodes 112, to maintaink-safety by reflecting the part of a projection that is being written toat least at least one of the external computing nodes 112 operating as amirror node for the computing node 201A, and to write to a loggingappliance corresponding to at least one of the external computing nodes112.

In the computing node system 200B of FIG. 2B, the computing node 201Bcomprises the processor 102 and a coherency fabric 222 with thereflective memory bridge 110 integrated therein. In the computing node201B, the coherency fabric 222 enables reflective memory operationsinvolving the processor 102, the local memory 104, and the externalcomputing nodes 112. Further, the database 204 of the computing node201B is able to request that the reflected memory controller 110 performreflective memory operations to communicate data, logs, commands,two-phase commits, and/or k-safety mirroring projections to at least oneof the external computing nodes 112 as described herein.

In the computing node system 200C of FIG. 2C, the computing node 201Ccomprises the processor 102 and a coherency fabric 232 with a nodecontroller 234 having the reflective memory bridge 110 integratedtherein. Without limitation to other examples, the node controller 234may operate in accordance with a quickpath interconnect (QPI)architecture. In the computing node 201C, the coherency fabric 232enables reflective memory operations involving the processor 102, thelocal memory 104, and the external computing nodes 112. Further, thedatabase 204 of the computing node 201C is able to request that thereflected memory controller 110 perform reflective memory operations tocommunicate data, logs, commands, two-phase commits, and/or k-safetymirroring projections to at least one of the external computing nodes112 as described herein.

In the computing node system 200D of FIG. 20, the computing node 201Dcomprises the processor 102 and a coherency fabric 242 with a PCI card244 having the reflective memory bridge 110 integrated therein. In thecomputing node 201D, the coherency fabric 232 enables reflective memoryoperations involving the processor 102, the local memory 104, andexternal computing nodes 112. Further, the database 204 of the computingnode 201D is able to request that the reflected memory controller 110perform reflective memory operations to communicate data, logs,commands, two-phase commits, and/or k-safety mirroring projections to atleast one of the external computing nodes 112 as described herein.

In the computing node systems 200A-200D of FIGS. 2A-2D, the externalcomputing nodes 112 may implement any of the reflective memory bridgeconfigurations described herein. The position of the reflective memorybridge 110 for a particular computing node (e.g., computing node 101,computing nodes 201A-201D, and computing nodes 112) may vary accordingto considerations such as implementation timing, implementationcomplexity, spacing requirements, and/or component availability.

FIG. 3 shows an example of an integrated circuit 300 in accordance withthe disclosure. The integrated circuit 300 may correspond to, forexample, a processor, a node controller, a PCI card, or a coherencyfabric component. As shown, the integrated circuit 300 comprises thereflective memory bridge 110, which is shown to specify for eachincoming region: a name for the region (matched to incoming regions orempty if this region is not active), physical addresses that comprisethe region, and a set of nodes which may reflect to this region (thenodes may be checked to reduce errors). Further, the reflective memorybridge 110 of the integrated circuit 300 may specify for each outgoingregion: a name for the region (if empty this region is not active),physical addresses that contribute to the region, and a set of nodesthat will receive updates to this region. As needed, the incomingregions or outgoing regions specified for the reflective memory bridge110 of the integrated circuit 300 may be updated by a user oradministrator.

FIG. 4 shows an example of a monitoring system 400 for a computing nodein accordance with the disclosure. The monitoring system 400 is able todetect when an interconnect for reflective memory operations becomesunavailable. As shown, the monitoring system 400 may comprise a buffer402 that receives and releases writes of an outgoing region (e.g., theoutgoing region 108) of local memory assigned for reflective memoryoperations. The monitoring system 400 also comprises a memory mappedregister 404 that maintains a count 406 based on incrementing the count406 when the buffer 402 receives a write from an outgoing region andbased on decrementing the count 406 when the buffer 402 releases a writefor transmission to an external computer node. If the count 406 exceedsa predetermined threshold (e.g., a fill level for the buffer 402 exceedsa predetermined threshold), an interrupt or notification is sent fromthe memory mapped register 404 to an interconnect controller 408 toreport an unavailable interconnect.

FIG. 5 shows an example of another monitoring system 500 for a computingnode in accordance with the disclosure. The monitoring system 500 isable to detect when a memory region for reflective memory operationsbecomes unavailable. As shown, the monitoring system 500 may comprise amemory mapped interface 504 with timeoutierror logic 506 to assert atimeout or error based on the availability information. The memorymapped interface 504 may, for example, support remote read/exchangeconstructs or remote read/increment constructs with timeout or errorindications provided by the timeoutierror logic 510 when an outgoingregion or an incoming region for reflective memory operations isunavailable. In such case, an interrupt or notification is sent from thememory mapped interface 504 to an interconnect controller 508 to reportan unavailable memory region. The unavailability of a memory region orinterconnect may occur for several reasons. For example, the receivermight crash causing the buffer to fill, an interconnect may have aglitch causing it to stop communications with an external node or nodes,or human error, (e.g., someone kicks out a power or network cable) maycause a memory region or interconnect to be unavailable.

FIG. 6 shows an example of various components of a computer system ornode 600 in accordance with the disclosure. The computer system 600 mayperform various operations to support the reflective memory operationsdescribed herein. The computer system 600 may correspond to componentsof the computing node 101, components of the computing node 201A-201D,or components of the external computing nodes 112 described herein.

As shown, the computer system 600 includes a processor 602 (which may bereferred to as a central processor unit or CPU) that is in communicationwith memory devices including secondary storage 604, read only memory(ROM) 606, random access memory (RAM) 608, input/output (I/O) devices610, and network connectivity devices 612. The processor 602 may beimplemented as one or more CPU chips.

It is understood that by programming and/or loading executableinstructions onto the computer system 600, at least one of the CPU 602,the RAM 608, and the ROM 606 are changed, transforming the computersystem 600 in part into a particular machine or apparatus having thenovel functionality taught by the present disclosure, In the electricalengineering and software engineering arts functionality that can beimplemented by loading executable software into a computer can beconverted to a hardware implementation by well-known design rules.Decisions between implementing a concept in software versus hardwaretypically hinge on considerations of stability of the design and numbersof units to be produced rather than any issues involved in translatingfrom the software domain to the hardware domain. For example, a designthat is still subject to frequent change may be implemented in software,because re-spinning a hardware implementation is more expensive thanre-spinning a software design. Meanwhile, a design that is stable thatwill be produced in large volume may be preferred to be implemented inhardware, for example in an application specific integrated circuit(ASIC), because for large production runs the hardware implementationmay be less expensive than the software implementation. Often a designmay be developed and tested in a software form and later transformed, bywell-known design rules, to an equivalent hardware implementation in anapplication specific integrated circuit that hardwires the instructionsof the software. In the same manner as a machine controlled by a newASIC is a particular machine or apparatus, likewise a computer that hasbeen programmed and/or loaded with executable instructions may be viewedas a particular machine or apparatus.

The secondary storage 604 may be comprised of one or more disk drives ortape drives and is used for non-volatile storage of data and as anover-flow data storage device if RAM 608 is not large enough to hold allworking data. Secondary storage 604 may be used to store programs whichare loaded into RAM 608 when such programs are selected for execution.The ROM 606 is used to store instructions and perhaps data which areread during program execution. ROM 606 is a non-volatile memory devicewhich typically has a small memory capacity relative to the largermemory capacity of secondary storage 604. The RAM 608 is used to storevolatile data and perhaps to store instructions. Access to both ROM 606and RAM 608 is typically faster than to secondary storage 604. Thesecondary storage 604, the RAM 608, and/or the ROM 606 may be referredto in some contexts as computer readable storage media and/ornon-transitory computer readable media.

I/O devices 610 may include printers, video monitors, liquid crystaldisplays (LCDs), touch screen displays, keyboards, keypads, switches,dials, mice, track balls, voice recognizers, card readers, paper tapereaders, or other well-known input devices.

The network connectivity devices 612 may take the force of modems,modern banks, Ethernet cards, universal serial bus (USB) interfacecards, serial interfaces, token ring cards, fiber distributed datainterface (FDDI) cards, wireless local area network (WLAN) cards, radiotransceiver cards such as code division multiple access (CDMA), globalsystem for mobile communications (GSM), long-term evolution (LTE),worldwide interoperability for microwave access (WiMAX), and/or otherair interface protocol radio transceiver cards, and other well-knownnetwork devices. These network connectivity devices 612 may enable theprocessor 602 to communicate with the Internet or one or more intranets.With such a network connection, it is contemplated that the processor602 might receive information from the network, or might outputinformation to the network in the course of performing theabove-described method steps. Such information, which is oftenrepresented as a sequence of instructions to be executed using processor602, may be received from and outputted to the network, for example, inthe form of a computer data signal embodied in a carrier wave.

Such information, which may include data or instructions to be executedusing processor 602 for example, may be received from and outputted tothe network, for example, in the form of a computer data baseband signalor signal embodied in a carrier wave. The baseband signal or signalembedded in the carrier wave, or other types of signals currently usedor hereafter developed, may be generated according to several methodsknown to one skilled in the art. The baseband signal and/or signalembedded in the carrier wave may be referred to in some contexts as atransitory signal.

The processor 602 executes instructions, codes, computer programs,scripts which it accesses from hard disk, floppy disk, optical disk(these various disk based systems may all be considered secondarystorage 604), ROM 606, RAM 608, or the network connectivity devices 612.While only one processor 602 is shown, multiple processors may bepresent. Thus, while instructions may be discussed as executed by aprocessor, the instructions may be executed simultaneously, serially, orotherwise executed by one or multiple processors. Instructions, codes,computer programs, scripts, and/or data that may be accessed from thesecondary storage 604, for example, hard drives, floppy disks, opticaldisks, and/or other device, the ROM 606, and/or the RAM 608 may bereferred to in some contexts as non-transitory instructions and/ornon-transitory information.

In an embodiment, the computer system 600 may comprise two or morecomputers in communication with each other that collaborate to perform atask. For example, but not by way of limitation, an application may bepartitioned in such a way as to permit concurrent and/or parallelprocessing of the instructions of the application. Alternatively, thedata processed by the application may be partitioned in such a way as topermit concurrent and/or parallel processing of different portions of adata set by the two or more computers. In an embodiment, virtualizationsoftware may be employed by the computer system 600 to provide thefunctionality of a number of servers that is not directly bound to thenumber of computers in the computer system 600. For example,virtualization software may provide twenty virtual servers on fourphysical computers, In an embodiment, the functionality disclosed abovemay be provided by executing the application and/or applications in acloud computing environment. Cloud computing may comprise providingcomputing services via a network connection using dynamically scalablecomputing resources. Cloud computing may be supported, at least in part,by virtualization software. A cloud computing environment may beestablished by an enterprise and/or may be hired on an as-needed basisfrom a third party provider. Some cloud computing environments maycomprise cloud computing resources owned and operated by the enterpriseas well as cloud computing resources hired and/or leased from a thirdparty provider.

In an embodiment, some or all of the reflective memory bridgefunctionality disclosed above may be provided as a computer programproduct. The computer program product may comprise one or more computerreadable storage medium having computer usable program code embodiedtherein to implement the functionality disclosed above. The computerprogram product may comprise data structures, executable instructions,and other computer usable program code. The computer program product maybe embodied in removable computer storage media and/or non-removablecomputer storage media. The removable computer readable storage mediummay comprise, without limitation, a paper tape, a magnetic tape,magnetic disk, an optical disk, a solid state memory chip, for exampleanalog magnetic tape, compact disk read only memory (CD-ROM) disks,floppy disks, jump drives, digital cards, multimedia cards, and others.The computer program product may be suitable for loading, by thecomputer system 600, at least portions of the contents of the computerprogram product to the secondary storage 604, to the ROM 606, to the RAM608, and/or to other non-volatile memory and volatile memory of thecomputer system 600. The processor 602 may process the executableinstructions and/or data structures in part by directly accessing thecomputer program product, for example by reading from a CD-ROM diskinserted into a disk drive peripheral of the computer system 600.Alternatively, the processor 602 may process the executable instructionsand/or data structures by remotely accessing the computer programproduct, for example by downloading the executable instructions and/ordata structures from a remote server through the network connectivitydevices 612. The computer program product may comprise instructions thatpromote the loading and/or copying of data, data structures, files,and/or executable instructions to the secondary storage 604, to the ROM606, to the RAM 608, and/or to other non-volatile memory and volatilememory of the computer system 600.

In some contexts, the secondary storage 604, the ROM 606, and the RAM608 may be referred to as a non-transitory computer readable medium or acomputer readable storage media. A dynamic RAM embodiment of the RAM608, likewise, may be referred to as a non-transitory computer readablemedium in that while the dynamic RAM receives electrical power and isoperated in accordance with its design, for example during a period oftime during which the computer 600 is turned on and operational, thedynamic RAM stores information that is written to it. Similarly, theprocessor 602 may comprise an internal RAM, an internal ROM, a cachememory, and/or other internal non-transitory storage blocks, sections,or components that may be referred to in some contexts as non-transitorycomputer readable media or computer readable storage media.

In some examples, a non-transitory computer-readable storage medium maystore reflective memory bridge instructions 609 that, when executed,cause the processor 602 to assign an incoming region of a Racal memoryto at least one external computing node. The reflective memory bridgeinstructions 609, when executed, also may cause the processor 602 toassign an outgoing region of the local memory to at least one externalcomputing node. The reflective memory bridge instructions 609, whenexecuted, also may cause the processor 602 to perform reflective memoryoperations using the incoming region and the outgoing region. In someexamples, the reflective memory bridge instructions 609, when executed,may cause the processor 602 to store for an outgoing region: a name,physical addresses associated with the outgoing region, and a set ofexternal computing nodes that are to receive updates to the outgoingregion. Further, the reflective memory bridge instructions 609, whenexecuted, may cause the processor 602 to store for an incoming region: aname, physical addresses associating with incoming region, and a set ofexternal computing nodes that are to reflect to the incoming region.Further, the reflective memory bridge instructions 609, when executed,may cause the processor 602 to determine a number of writes from anoutgoing region that are still in a buffer by incrementing a count whena write from the outgoing region is observed and decrementing the countwhen a write is released from the buffer. Further, the reflective memorybridge instructions 609, when executed, may cause the processor 602 toperform other reflective memory bridge operations as described herein.

FIG. 7 shows an example of a method 700 in accordance with thedisclosure. The method 700 may be performed by a reflective memorybridge and/or other components of a computing node such as of thecomputing node 101, components of the computing node 201A-201D, orcomponents of the external computing nodes 112 described herein. Asshown, the method 700 comprises assigning an incoming region of a localmemory to at least one external computing node (block 702). At block704, the method 700 also comprises assigning an outgoing region of thelocal memory to at least one external computing node. At block 706, themethod 700 comprises performing reflective memory operations using theincoming region and the outgoing region.

In some examples, the method 700 may comprise additional or alternativesteps. For example, the method 700 may further comprise storing for anoutgoing region: a name, physical addresses associated with the outgoingregion, and a set of external computing nodes that are to receiveupdates to the outgoing region. Further, the method 700 also maycomprise storing for an incoming region: a name, physical addressesassociating with incoming region, and a set of external computing nodesthat are to reflect to the incoming region. Further, the method 700 alsomay comprise determining unavailability of an interconnect or a memoryregion used for reflective memory operations. Further, method 700 alsomay comprise performing other reflective memory bridge operations asdescribed herein.

The above discussion is meant to be illustrative of the principles andvarious examples of the present invention. Numerous variations andmodifications will become apparent to those skilled in the art once theabove disclosure is fully appreciated. It is intended that the followingclaims be interpreted to embrace all such variations and modifications.

1.-15. (canceled)
 16. A computing node, comprising: a reflective memorybridge to map to an incoming region of a local memory assigned to anexternal computing node and to map to an outgoing region of the localmemory assigned to an external computing node; and a memory mappedregister to count a number of writes from the outgoing region that arestill in a buffer by incrementing the count when a write from theoutgoing region is monitored and decrementing the count when the writeis released from the buffer; wherein a same counter of the memory mappedregister is incremented when the write from the outgoing region ismonitored and is decremented when the write is released from the buffer.17. The computing node of claim 16, wherein the reflective memory bridgeis part of a coherency fabric for the computing node and is integratedwith a component selected from a list consisting of the processor, anode controller for the computing node, a coherency bus for thecomputing node, and personal component interconnect (PCI) card for thecomputing node.
 18. The computing node of claim 16, wherein thereflective memory bridge specifies for the outgoing region: a name,addresses associated with the outgoing region, and a set of externalcomputing nodes that are to receive updates to the outgoing region. 19.The computing node of claim 16, wherein the reflective memory bridgespecifies for the incoming region: a name, addresses associated with theincoming region, and a set of external computing nodes that are toreflect to the incoming region.
 20. The computing node of claim 16,wherein stores from the reflective memory bridge are not reflected. 21.The computing node of claim 16, further comprising a memory mappedinterface that allows remote read/exchange constructs or remoteread/increment constructs.
 22. The computing node of claim 21, whereinthe memory mapped interface further transmits a timeout or errornotification to an interconnect controller when the outgoing region orthe incoming region is determined to be unavailable.
 23. The computingnode of claim 21, further comprising a database that utilizes thereflective memory bridge to communicate commands or results to at leastone external computing node.
 24. The computing node of claim 21, furthercomprising a database that utilizes the reflective memory bridge todistribute logs, updates, and new data to at least one externalcomputing node.
 25. The computing node of claim 21, further comprising adatabase that utilizes the reflective memory bridge to communicateinformation for two-phase commits to at least one other computing node.26. The computing node of claim 21, further comprising a database thatutilizes the reflective memory bridge to maintain k-safety by reflectingat least part of a projection being written to a mirror node.
 27. Acomputing node comprising: a reflective memory bridge to map to anincoming region of a local memory assigned to an external computing nodeand to map to an outgoing region of the local memory of the local memoryassigned to an external computing node; a database that utilizes thereflective memory bridge to communicate information for two-phasecommits to another computing node; and a monitoring circuit to monitorthat an interconnect between the computing node and the external node isoperable by tracking a number of writes from the outgoing region thatare still in a buffer, wherein a same counter is incremented when awrite is monitored from the outgoing region and is decremented when thewrite is released from the buffer.
 28. The computing node of claim 27,wherein the reflective memory bridge specifies for the outgoing region:a name, addresses associated with the outgoing region, and a set ofexternal computing nodes that are to receive updates to the outgoingregion.
 29. The computing node of claim 27, wherein the reflectivememory bridge specifies for the incoming region: a name, addressesassociated with the incoming region, and a set of external computingnodes that are to reflect to the incoming region.
 30. The computing nodeof claim 27, further comprising a memory mapped interface that allowsremote read/exchange constructs or remote read/increment constructs. 31.The computing node of claim 30, wherein the memory mapped interfacefurther transmits a timeout or error notification to an interconnectcontroller when the outgoing region or the incoming region is determinedto be unavailable.
 32. A computing node comprising: a reflective memorybridge to map to an incoming region of a local memory assigned to anexternal computing node and to map to an outgoing region of the localmemory of the local memory assigned to an external computing node; adatabase that utilizes the reflective memory bridge to maintain k-safetyby reflecting at least part of a projection being written to a mirrornode; and a monitoring circuit to monitor that an interconnect betweenthe computing node and the external node is operable by tracking anumber of writes from the outgoing region that are still in a buffer,wherein a same counter is incremented when a write is monitored from theoutgoing region and is decremented when the write is released from thebuffer.
 33. The computing node of claim 32, wherein the reflectivememory bridge specifies for the outgoing region: a name, addressesassociated with the outgoing region, and a set of external computingnodes that are to receive updates to the outgoing region, and whereinthe reflective memory bridge specifies for the incoming region: a name,addresses associated with the incoming region, and a set of externalcomputing nodes that are to reflect to the incoming region.
 34. Thecomputing node of claim 32, further comprising a memory mapped interfacethat allows remote read/exchange constructs or remote read/incrementconstructs.
 35. The computing node of claim 32, wherein the memorymapped interface further transmits a timeout or error notification to aninterconnect controller when the outgoing region or the incoming regionis determined to be unavailable.